biological science
Biothreat Benchmark Generation Framework for Evaluating Frontier AI Models II: Benchmark Generation Process
Ackerman, Gary, Kallenborn, Zachary, Wetzel, Anna, Peterson, Hayley, LaTourette, Jenna, Shoemaker, Olivia, Behlendorf, Brandon, Almakki, Sheriff, Clifford, Doug, Sheinbaum, Noah
The potential for rapidly-evolving frontier artificial intelligence (AI) models, especially large language models (LLMs), to facilitate bioterrorism or access to biological weapons has generated significant policy, academic, and public concern. Both model developers and policymakers seek to quantify and mitigate any risk, with an important element of such efforts being the development of model benchmarks that can assess the biosecurity risk posed by a particular model. This paper, the second in a series of three, describes the second component of a novel Biothreat Benchmark Generation (BBG) framework: the generation of the Bacterial Biothreat Benchmark (B3) dataset. The development process involved three complementary approaches: 1) web-based prompt generation, 2) red teaming, and 3) mining existing benchmark corpora, to generate over 7,000 potential benchmarks linked to the Task-Query Architecture that was developed during the first component of the project. A process of de-duplication, followed by an assessment of uplift diagnosticity, and general quality control measures, reduced the candidates to a set of 1,010 final benchmarks. This procedure ensured that these benchmarks are a) diagnostic in terms of providing uplift; b) directly relevant to biosecurity threats; and c) are aligned with a larger biosecurity architecture permitting nuanced analysis at different levels of analysis.
- North America > United States (0.14)
- Asia > North Korea (0.04)
- Asia > China > Hong Kong (0.04)
- Africa > Cameroon > Gulf of Guinea (0.04)
- Workflow (0.94)
- Research Report > Experimental Study (0.68)
- Health & Medicine (1.00)
- Government > Military (1.00)
- Law Enforcement & Public Safety > Crime Prevention & Enforcement (0.93)
- (2 more...)
#AAAI2025 workshops round-up 3: Neural reasoning and mathematical discovery, and AI to accelerate science and engineering
In this series of articles, we're publishing summaries with some of the key takeaways from a few of the workshops held at the 39th Annual AAAI Conference on Artificial Intelligence (AAAI 2025). Recent progress in Sphere Neural Networks demonstrates various possibilities for neural networks to achieve symbolic-level reasoning. This workshop aimed to reconsider various problems and discuss walk-round solutions in the two-way street commingling of neural networks and mathematics. This workshop brought together researchers from artificial intelligence and diverse scientific domains to address new challenges towards accelerating scientific discovery and engineering design. This was the fourth iteration of the workshop, with the theme of AI for biological sciences following previous three years' themes of AI for chemistry, earth sciences, and materials/manufacturing respectively.
Collective Innovation in Groups of Large Language Models
Nisioti, Eleni, Risi, Sebastian, Momennejad, Ida, Oudeyer, Pierre-Yves, Moulin-Frier, Clément
Human culture relies on collective innovation: our ability to continuously explore how existing elements in our environment can be combined to create new ones. Language is hypothesized to play a key role in human culture, driving individual cognitive capacities and shaping communication. Yet the majority of models of collective innovation assign no cognitive capacities or language abilities to agents. Here, we contribute a computational study of collective innovation where agents are Large Language Models (LLMs) that play Little Alchemy 2, a creative video game originally developed for humans that, as we argue, captures useful aspects of innovation landscapes not present in previous test-beds. We, first, study an LLM in isolation and discover that it exhibits both useful skills and crucial limitations. We, then, study groups of LLMs that share information related to their behaviour and focus on the effect of social connectivity on collective performance. In agreement with previous human and computational studies, we observe that groups with dynamic connectivity out-compete fully-connected groups. Our work reveals opportunities and challenges for future studies of collective innovation that are becoming increasingly relevant as Generative Artificial Intelligence algorithms and humans innovate alongside each other.
- Oceania (0.04)
- North America > United States > New York > New York County > New York City (0.04)
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.04)
- (2 more...)
Cultural evolution in populations of Large Language Models
Perez, Jérémy, Léger, Corentin, Ovando-Tellez, Marcela, Foulon, Chris, Dussauld, Joan, Oudeyer, Pierre-Yves, Moulin-Frier, Clément
Research in cultural evolution aims at providing causal explanations for the change of culture over time. Over the past decades, this field has generated an important body of knowledge, using experimental, historical, and computational methods. While computational models have been very successful at generating testable hypotheses about the effects of several factors, such as population structure or transmission biases, some phenomena have so far been more complex to capture using agent-based and formal models. This is in particular the case for the effect of the transformations of social information induced by evolved cognitive mechanisms. We here propose that leveraging the capacity of Large Language Models (LLMs) to mimic human behavior may be fruitful to address this gap. On top of being an useful approximation of human cultural dynamics, multi-agents models featuring generative agents are also important to study for their own sake. Indeed, as artificial agents are bound to participate more and more to the evolution of culture, it is crucial to better understand the dynamics of machine-generated cultural evolution. We here present a framework for simulating cultural evolution in populations of LLMs, allowing the manipulation of variables known to be important in cultural evolution, such as network structure, personality, and the way social information is aggregated and transformed. The software we developed for conducting these simulations is open-source and features an intuitive user-interface, which we hope will help to build bridges between the fields of cultural evolution and generative artificial intelligence.
- North America > United States > New Mexico > Los Alamos County > Los Alamos (0.04)
- North America > United States > Illinois > Cook County > Chicago (0.04)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.04)
- (5 more...)
- Information Technology > Artificial Intelligence > Representation & Reasoning > Agents (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning > Generative AI (0.34)
Artificial intelligence technologies to support research assessment: A review
Kousha, Kayvan, Thelwall, Mike
This literature review identifies indicators that associate with higher impact or higher quality research from article text (e.g., titles, abstracts, lengths, cited references and readability) or metadata (e.g., the number of authors, international or domestic collaborations, journal impact factors and authors' h-index). This includes studies that used machine learning techniques to predict citation counts or quality scores for journal articles or conference papers. The literature review also includes evidence about the strength of association between bibliometric indicators and quality score rankings from previous UK Research Assessment Exercises (RAEs) and REFs in different subjects and years and similar evidence from other countries (e.g., Australia and Italy). In support of this, the document also surveys studies that used public datasets of citations, social media indictors or open review texts (e.g., Dimensions, OpenCitations, Altmetric.com and Publons) to help predict the scholarly impact of articles. The results of this part of the literature review were used to inform the experiments using machine learning to predict REF journal article quality scores, as reported in the AI experiments report for this project. The literature review also covers technology to automate editorial processes, to provide quality control for papers and reviewers' suggestions, to match reviewers with articles, and to automatically categorise journal articles into fields. Bias and transparency in technology assisted assessment are also discussed.
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.27)
- Europe > Italy (0.24)
- Oceania > Australia (0.24)
- (24 more...)
- Research Report > New Finding (1.00)
- Research Report > Experimental Study (1.00)
- Overview (1.00)
- Media > Publishing (1.00)
- Health & Medicine > Therapeutic Area (1.00)
- Health & Medicine > Pharmaceuticals & Biotechnology (1.00)
- Education (1.00)
Evolved Open-Endedness in Cultural Evolution: A New Dimension in Open-Ended Evolution Research
Borg, James M., Buskell, Andrew, Kapitany, Rohan, Powers, Simon T., Reindl, Eva, Tennie, Claudio
The goal of Artificial Life research, as articulated by Chris Langton, is "to contribute to theoretical biology by locating life-as-we-know-it within the larger picture of life-as-it-could-be" (1989, p.1). The study and pursuit of open-ended evolution in artificial evolutionary systems exemplifies this goal. However, open-ended evolution research is hampered by two fundamental issues; the struggle to replicate open-endedness in an artificial evolutionary system, and the fact that we only have one system (genetic evolution) from which to draw inspiration. Here we argue that cultural evolution should be seen not only as another real-world example of an open-ended evolutionary system, but that the unique qualities seen in cultural evolution provide us with a new perspective from which we can assess the fundamental properties of, and ask new questions about, open-ended evolutionary systems, especially in regard to evolved open-endedness and transitions from bounded to unbounded evolution. Here we provide an overview of culture as an evolutionary system, highlight the interesting case of human cultural evolution as an open-ended evolutionary system, and contextualise cultural evolution under the framework of (evolved) open-ended evolution. We go on to provide a set of new questions that can be asked once we consider cultural evolution within the framework of open-ended evolution, and introduce new insights that we may be able to gain about evolved open-endedness as a result of asking these questions.
- Europe > United Kingdom > England > Oxfordshire > Oxford (0.28)
- Europe > United Kingdom > England > Cambridgeshire > Cambridge (0.28)
- North America > United States > Massachusetts > Middlesex County > Cambridge (0.14)
- (6 more...)
- Overview (0.68)
- Instructional Material (0.46)
New AI-based tool helps clinicians understand and better predict adverse effects of COVID-19
The symptoms and side effects of Covid-19 are scattered across a diagnostic spectrum. Some patients are asymptomatic or experience a mild immune response, while others report significant long-term illnesses, lasting complications, or suffer fatal outcomes. Three researchers from the Georgia Institute of Technology and one from Emory University are trying to help clinicians sort through these factors and spectrum of patient outcomes by equipping healthcare professionals with a new "decision prioritization tool." The team's new artificial intelligence-based tool helps clinicians understand and better predict which adverse effects their Covid-19 patients could experience, based on comorbidities and current side effects -; and, in turn, also helps suggest specific Food and Drug Administration-approved (FDA) drugs that could help treat the disease and improve patient health outcomes. The researcher's latest findings are the focus of a new study published October 21 in Scientific Reports. The team's new methodology, or tool, is called MOATAI-VIR (Mode Of Action proteins & Targeted therapeutic discovery driven by Artificial Intelligence for VIRuses.
AI tool pairs protein pathways with clinical side effects, patient comorbidities to suggest targeted Covid treatments
The symptoms and side effects of Covid-19 are scattered across a diagnostic spectrum. Some patients are asymptomatic or experience a mild immune response, while others report significant long-term illnesses, lasting complications, or suffer fatal outcomes. Three researchers from the Georgia Institute of Technology and one from Emory University are trying to help clinicians sort through these factors and spectrum of patient outcomes by equipping healthcare professionals with a new "decision prioritization tool." The team's new artificial intelligence-based tool helps clinicians understand and better predict which adverse effects their Covid-19 patients could experience, based on comorbidities and current side effects -- and, in turn, also helps suggest specific Food and Drug Administration-approved (FDA) drugs that could help treat the disease and improve patient health outcomes. The researcher's latest findings are the focus of a new study published October 21 in Scientific Reports.
- North America > United States (0.70)
- Europe > France (0.05)
- Asia > China (0.05)
Using Artificial Intelligence To Discover New Drug Treatments: Science Next
In the fields of medicine and pharmacology, drugs have historically been discovered either by identifying active ingredients from traditional remedies, or by serendipity, similar to how penicillin was discovered in 1928. But modern scientists have found an alternative method that makes the hunt for new pharmaceuticals quicker, cheaper and more effective. AI, or artificial intelligence, can be found in everything from chess-playing computers, to self-driving cars, to the maps application on your phone when it's calculating directions. And AI has found a new home: discovering new drugs. It currently takes on average 10 years and over $2 billion to create a new drug and get it approved.
The Bio Revolution: Innovations transforming economies, societies, and our lives
A confluence of advances in biological science and accelerating development of computing, automation, and artificial intelligence is fueling a new wave of innovation. This Bio Revolution could have significant impact on economies and our lives, from health and agriculture to consumer goods, and energy and materials. Some innovations come with profound risks rooted in the self-sustaining, self-replicating, and interconnected nature of biology that argue for a serious and sustained debate about how this revolution should proceed. Accidents can have major consequences--and, especially if used unethically or maliciously, manipulating biology could become a Pandora's box that, once opened, unleashes lasting damage to the health of humans, ecosystems, or both. The risks are particularly acute because many of the materials and tools are relatively cheap and accessible. Moreover, tackling these risks is complicated by a multiplicity of jurisdictional and cultural value systems, which makes collaboration and coordination across countries difficult. While the impact of COVID 19 was still unfolding at the time of writing in April 2020, bio innovations had been deployed to aid the response.